| min rank | avg. rank | sentence |
|---|---|---|
| 6377 | 13804.0000 | Architect: Stone & Webster Engineering Co. |
| 6374 | 33945.2857 | Pyrat XO Reserve Rum (Antigua): Amber colored. |
| 5087 | 35622.0000 | Electronic Nicotine Delivery System Products. |
| 4287 | 53540.5714 | Aller-Ease Cotton Hypoallergenic Allergy Protection Body Pillow,… |
| 3854 | 16250.5000 | Chili beef stuffing bake, corn, fruit, cornbread, milk. |
| 3624 | 58394.8182 | & Easy Access Subd., Spring Village & Meadowland Village, Cuanos Elem. |
| 3426 | 33991.1429 | ALCF Data Science Program (ADSP): https://www. |
| 3234 | 37949.6250 | Watch Official Music Video: Sarkodie – Biibiba ft. |
| 2929 | 40602.5714 | Mixed Reactions Trail $177m Ogoni Clean-Up Fund - :::. |
| 2893 | 13036.2500 | Bestman Games hosts 2nd Annual Children Finance Fair » |
| 2643 | 36403.7000 | Captain 9th Battalion Lancashire Fusiliers 11th Division. |
| 2624 | 24972.0000 | Meyer praised McSorley’s competitive edge. |
| 2520 | 30141.8750 | Labor History Film Series Shows Democratizing Force… |
| 2255 | 42273.1765 | Italian fashion designer Maria Grazia Chiuri attends Christian Dior's ready-to-wear fall/winter 2018/2019 fashion collection. |
| 2209 | 25780.7000 | Think plant milk, plant protein, plant-based diets — vegetarian, vegan, flexitarian. |
| 2107 | 17389.7778 | Imagine Dragons: 'Natural' Stream, Lyrics & Download - Listen Here! |
| 2071 | 34823.6000 | Fans Who Harassed Kelly Marie Tran Are “M… |
| 2033 | 8154.8750 | Eleanor M. & Herbert D. Katz Family Foundation, Inc. |
| 1912 | 33389.8889 | Series Star Bret Harrison Wants Your Foot Warmers! |
| 1875 | 49994.9474 | Super Moisture ShampooNot Your Mother’s Natural Matcha Green Tea & Wild Apple Blossom Nutrient Rich Butter Masque. |
| 1751 | 36053.4286 | Fingerprint Punch Card Machine Market https://www. |
| 1748 | 26955.1250 | Should Canadian opera fix Mozart’s casual racism? |
| 1656 | 38851.0000 | Teen Serve Fundraiser, multi-family garage sale. |
| 1529 | 18612.9000 | Petersburg, Los Angeles, Chicago, Silicon Valley, Raleigh-Durham, Minneapolis/St. |
| 1429 | 53646.3000 | Nwoko Vs Nwaoboshi: Who Does Anioma Red Cap Fit? |
| 1427 | 21974.2500 | Winter weather returns, causing delays, closures – WisconsinReport. |
| 1420 | 25155.1818 | ATTLEBORO, Mass. (WPRI) — Earl Goff loved Bliss’ peach ice cream. |
| 1373 | 9686.3000 | Big god…Money, F*ck boy…s*x, Book boy. |
| 1204 | 20209.0000 | Eye contact, earnest conversation, connection. |
| 1183 | 38847.9091 | Is Ravi Shastri dating Bollywood actress Nimrat Kaur? |
In contrast to subsection 4.5.2.1 we now search for sentences consisting of rare words only. The sentences are ordered by the rank of the most frequent word in a sentence. In the table, we see the corresponding sentences with a minimum length of 40 characters.
The sentences are forced not to contain any everyday word. As a consequence, we get either sentences of some very reduced structure or sentences in some foreign language. Hence, the data are useful for the evaluation of the preprocessing, especially language detection.
select min(w_id)-100 as m, avg(w_id)-100 as a, s.sentence from sentences s, inv_w i where s.s_id=i.s_id and length(sentence)>40 and i.w_id>100 group by s.s_id order by m desc limit 30;
Should we remove the sentences having its least frequent word above some threshold?
4.5.2.1 Maximum word rank in sentence
4.5.2.2 Average word rank in sentence
4.5.2.4 Sentences consisting of many low frequency words II
4.5.2.5 Sentences consisting of short words only I
4.5.2.6 Sentences consisting of short words only II
4.5.2.7 Sentences consisting of long words only I
4.5.2.8 Sentences consisting of long words only II